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Grade Equivalent and IRT Representations of Growth 



E. Matthew Schulz and W. Alan Nicewander 



Abstract 



It has long been a part of psychometric lore that the variance of children's scores 
on cognitive tests increases with age. This "increasing-variance phenomenon" was first 
observed on Binet's intelligence measures in the early 1900's. An important detail in this 
matter is the fact that developmental scales based on age or grade have served as the 
medium for demonstrating the increasing-variance phenomenon. Recently, 
developmental scales based on item response theory (IRT) have shown constant or 
decreasing variance of measures of achievement with increasing age. This discrepancy 
is of practical and theoretical importance. Conclusions about the effects of variables on 
growth in achievement will depend on the metric chosen. In this study, growth in the 
mean of a latent educational achievement variable is assumed to be a negatively- 
accelerated function of grade; within-grade variance is assumed to be constant across 
grade, and observed test scores are assumed to follow an IRT model. Under these 
assumptions, the variance of grade equivalent scores increases markedly. Perspective 
on this phenomenon is gained by examining longitudinal trends in centimeter and age 
equivalent measures of height. 
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Grade equivalent and IRT Representations of Growth 

The use of item response theory as a model for cognitive test data has recently 
introduced some controversial discrepancies concerning trends in the variability of 
mental traits with age (Hoover, 1984a,b, Burket, 1984; 1988; Yen, 1988; Phillips and 
Clarizio, 1988a, b; Clemans, 1993). There is a strong, mutual reinforcement between the 
popular notion that variance of cognitive skills increases with age, and the fact that 
grade equivalent and Thurstonian scales have traditionally confirmed this trend. A 
trend of increasing variance is consistent with the common-sense notion that above- 
average students continue to develop at a faster rate than below average students. 
However, when IRT scales are constructed from the same or comparable data used to 
construct grade equivalent and Thurstonian scales, IRT variability remains constant or 
even decreases (Yen, 1986; Schulz, Shen, and Wright, 1990; Lee and Wright, 1992; Bock, 
1983). 

Differences in the growth rate of mean or median achievement also exist between 
metrics. The defining characteristic of a grade equivalent scale is that median 
achievement in the norm group increases at a constant rate of one unit per year. 
Thurstonian and IRT scalings of educational achievement data generally show increases 
in the mean to be negatively accelerated with grade (Yen, 1986; Schulz, et al., 1990; Lee 
and Wright, 1992). 

Differences in growth trends have practical importance in research on educational 
achievement. In a longitudinal study of the effects of schools and other higher-level 
variables on change in student's educational achievement, grade equivalent and IRT 
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metrics led to strikingly different representations of individual differences in growth 
trends among students (Seltzer, Frank and Bryk, 1994). These investigators concluded 
that choice of metric can influence decisions about the efficacy of educational programs. 

The problem of choosing a scale for research on growth in educational 
achievement is complicated by the arbitrary nature of scales. Educational and cognitive 
tests do no more than order levels of cognitive performance. One cannot pose questions 
about trends in variability and rates of growth until test results are put on a metric scale. 
The only nonarbitrary criterion of a scale is that it preserve the ordering of performance 
in the test data. Two scales that are equally acceptable from this perspective can lead 
to opposite conclusions about trends in variability and rates of growth (Braun, 1988). 
Zwick (1992) gives an example in which a difference of increasing variance is converted 
to one of decreasing variance by an order-preserving transformation. 

One aim of this paper is to show how differences in IRT and grade equivalent 
growth trends stem from differences in the scaling models. Yen (1986) and Schulz (1990) 
have pointed out that grade equivalent variance is bound to increase if an alternative 
order preserving metric shows a pattern of constant within-grade variance and 
negatively accelerated growth in the mean. To demonstrate this point here, growth 
trends of constant variability across grades and negatively accelerated growth in the 
mean on an IRT (theta) metric are assumed, and test data is assumed to fit a given IRT 
model (see Equation (1) in the following section). A grade equivalent scale is then 
constructed in order to illustrate the trend of increasing variability on the grade 
equivalent scale. 
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In the standard procedure for constructing grade equivalent scales (Petersen, 
Kolen, and Hoover, 1989), grade equivalents have a one-to-one correspondence with true 
scores on grade-level tests administered to students within grade and with true scores 
on a scaling test administered to all students. Trends in the distribution of number 
correct scores on such tests will be examined in this study, but will not be used to 
construct the grade equivalent scale. Instead, thetas will be mapped directly into grade 
equivalent scores because there is a one-to-one correspondence between thetas and 
number correct true scores in the IRT model. The purpose of examining trends in 
number correct scores is to demonstrate their relationship to trends in the theta metric. 

Two methods will be used to map IRT ability parameters directly to grade 
equivalents. One method uses quadratic regression, and is suitable when the growth in 
mean achievement on an IRT scale exhibits a simple quadratic trend, as will be assumed 
in this study. A more general, but less exact method, called integer-assignment, maps 
theta values to the most probable grade (integer, grade equivalent value), according to 
the relative density of the assumed within-grade theta distributions. The later method 
is considered more suitable when growth in the mean is not a simple quadratic function 
over grade. Both methods are expected to yield comparable results. 

To add perspective on the meaning of growth trends in either metric, an analogy 
to growth in a physical characteristic, height, is developed. Growth in physical 
characteristics has long served as a model for growth in mental traits (Bloom, 1966; Bock, 
1989). As will be seen, centimeter measures of height, grouped by age, show trends of 
decreasing, as well as increasing variance with age and nonlinear rates of growth in the 
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mean. These trends provide a basis for interpreting similar trends in IRT measures of 
educational achievement. The analogy is extended further by mapping centimeter 
measures of height into age equivalent scores. The relationship between age equivalent 
and centimeter growth trends in height is comparable to the relationship between grade 
equivalent and IRT growth trends in educational achievement. 

Methods 

Assumptions 

Let 0 represent a latent scale of achievement, and let the probability of a correct 
answer to a multiple choice achievement test item be the following logistic function of 
0 : 



P(0) = 0.2 



0.8 



(1) 



1 + exp(l. 7(b-0)) 

b is the 0-coordinate of the point of inflection of the regression line of P(0) on 0 for the 
given item. 

Let j represent grade, and let the distribution of student achievement within grade 
j be N(^ ;0 ,a ;0 2 ), where 



P;0 = 



-.133 + 



35./ - j 2 
30 



;'= 1 , 2 ,..., 12 . 



(2) 



and a ;0 =1 for all j. The within-grade mean and standard deviation of 0 are plotted by 
grade in Figure 1. The negatively accelerated rate of growth in the mean in Equation 
(2) is a reasonable approximation to observed trends (Yen, 1986; Schulz, et al., 1990; Lee, 
et al., 1992). Constant within-grade variance across grades is also an approximation to 
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reported IRT trends (Schulz, et al., 1990; Lee, et al., 1992). Marked decreases in IRT 
variability over grade (Yen, 1986) are not taken into account here because they may have 
been due, in part, to problems with estimation procedures (Williams, et al., 1995). 




Figure 1. Hypothetical trends in the mean and standard deviation of a latent educational achievement variable 
(0). The dashed line shows linearity for purposes of comparison to the nonlinear trend in the mean. 



Test Specifications and Trends in Number Correct Scores 

Grade-level Tests. Let the grade-level test for grade; consist of n items with b=PjQ, 
let Py(0) be the probability that a student with 0=0 gets a grade-j item correct, let S 
represent the number correct score on the grade-level test, and let /(0 1 j) represent the 
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theta density function within grade j. The within-grade mean of S is constant across 
grades: 



E[S |/]= p. s = U S = n JPj(e)/(e|;)de . - «) 

— oo 

Constancy in the ]ij S over grades is due, in this case, to the relative difficulty of the 
grade-level tests being the same across grades (b=j/y 0 ), and to the assumption that the 
within-grade distributions of 0 are identical across grades except for the mean. The 
within-grade variance of S is also constant across grades: 

Var(S|;> = 0? s = c J s = E e [Var(S|9)] * Var 0 (E[S|9]) (4 > 

where 



Var 0 (E[S|9]) = j (n P/e)) 2 /(9 |;)d9 - u 2 (S) 



(5) 



and 



E 0 [Var(S|9)] = n J P ; (9)(l -P/6))/(e |/)d6 . <« 

— OO 

rs _ 

<5j S is constant across grades for the same reasons ji s is constant across grades. Given 
the assumption of model fit, normally distributed 0 within grade, and b=fi(Q I j) for 
grade-j items, the trend in Gy S versus grade will have the same sign as the trend in Oy 0 
versus grade. Both trends have zero slope in the present case due to the defined 
condition o^ -l for all j. 
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Scaling Test. Let the scaling test consist of k items from each of the grade-level 
tests, and let M represent the number right score on the scaling test. The within-grade 
mean of M is: 
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e[m|j] = = kf C p/e)]f(9|/)de . 

— OO /— 1 



(7) 



The within-grade variance of M, C jM' follows the general form of Equation (4), where 
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Var@(E[My!0]) = f [£ (fcp / (e)) 2 ]f(e |;)de - 

-oo ;=i 



( 8 ) 



and 



oo 12 

EglVarfMyle)] = k f [£ P/e)/i-P/e))]«e |/)de . < 9 > 

-oo /= 1 

Figure 2 shows a plot of ji: M and Oj M when k= 10. There is a slight S-shape in the 
plot of and a bell-shape to the trend of Gj M . Both trends are connected to the fact 
that the difference between the number correct scores of any given pair of students is 
relatively small when a test is very easy or very hard for both students. Students for 
whom a given test is very easy or very hard are said to be performing near the test's 
ceiling or floor. Floor and ceiling effects of the present scaling test include the apparent 
shrinkage in within-grade variability at upper and lower grades, and the decline in the 
difference between means of adjacent grade groups at upper and lower grades. With 
the help of such effects, and the right set of test specifications, one can create virtually 
any trend in the distributions of number correct scores. 
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Grade 



Figure 2. The mean and standard deviation, by grade, of the number correct score on a hypothetical scaling 
test administered to all students (see text for test specifications). 



Mapping IRT Values Into Grade Equivalents 

Let G denote the grade-equivalent variable, let H [ (g) be the cumulative density of 
grade-equivalent scores within grade j, and let F ; (0) be the cumulative density of 0 
within grade j. Because true scores on the scaling test are a one-to-one function of 0, 
and 0 is normally distributed within grade, H Ug) is defined for g=l,...,12, exactly as in 
the true score procedure (Petersen, et al., 1989) by 
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( 10 ) 



That is, the grade-j percentile ranks of grade equivalent values g=l,...,12 are, respectively, 
the grade-j percentile ranks of the grade )' median 0s, where j'=l,...,12. The points 
labeled "Medians" in Figure 3 are consistent with Equation (10). These "median-by- 



is mapped directly to G. 

The use of interpolation between the median anchor points in Figure 3, to map 
0 to G, is equivalent to mapping true scores on the grade-level tests to G as described 
by Petersen, et al., (1989). The procedures are equivalent because grade-level test true 
scores are one-to-one transforms of 0. 

Additional anchor points for mapping 0 to G are labeled "equal-density" in Figure 
3. These were computed as follows: Let 0 + 5 represent the 0 for which f(0 1 f) = f(0 ly'+l). 
If the within-grade distribution of 0 is normal with variance constant across grade, then 
Q. + 5 = (Fye+F(/+i)e)/2- That is, Qj + 5 is exactly half-way between and F(/+i)e- If within- 
grade variances are not equal, but 0 is normally distributed within each grade, then 0y + 5 
= (WjUjQ + Wy + i^(y + i)e) /( VVy-t- VVy + i), where Wj and Wy +1 are the within-grade standard 
deviation of 0 for, respectively, grades; and j+1. The open squares in Figure 3 represent 
the equal-density points. These have the coordinates (0y + 5 ,g=j+.5), j= 1,...,11. From visual 
inspection, these points are in the same trend line as the median-by-definition anchor 



Rather than interpolating between the anchor points in Figure 3, two analytical 
methods were used to map 0 into grade equivalents. 



definition" anchor points define j as the median grade-equivalent value for grade j if 0 
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Figure 3. Relationship between grade equivalent scores and achievement on the 0 metric. Coordinates of points 
designated 'medians' are the median grade equivalent and median 0 within grade. Coordinates of points 
designated 'equal density' are the lower median grade equivalent plus 0.5, and the 0 value that is equally 
likely to correspond to the lower or higher grade. The quadratic regression line was estimated using both 
'median' and ‘equal density' points. 



Mapping through Quadratic Regression. A quadratic regression of G on 0 was fit 
to the anchor points plotted in Figure 3. Median-by-definition anchor points, (PjQ,g=j), 
and equal-density anchor points, (0 ;+ 5 ,g=/+.5), yielded identical regression equations. 
The regression equation was quadratic with positive acceleration: 

G = 0.57 + 0.50 + O.O80 2 . UD 

This equation fit the points in Figure 3 very well (R 2 =.999). Let Equation (11) be 
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expressed in the following general form: 

g = p„ - M ♦ p 2 e 2 

Then the mean of G for any grade, j, is: 



( 12 ) 



E[G|;] = ]x jG = E[(P 0 + Pj0 + P 2 e 2 )!;] 



= p 0 + PjEIGl;] + P 2 E[e z | ;] 

= Pq + Pl^/0 + M^/e + a ;0 



( 13 ) 



The last line of (13) follows from the identity o- Q 2 = E[0 2 1 /'] - ^ e 2 . By using a Taylor 
series expansion, it can be shown that the grade-j variance of G is: 



2 2 



°/G ■ E[G I/] - \i jc 

= E[(p 0 + p,e * PjQ 2 ) 2 1;'] 



2 

H/G 



( 14 ) 



= 2p 2 a^ + dj e ( Pi + 2p 2 p 2 0 ) 



The grade-j median of G can be expressed as: 



Med{G\j) = p 2 p ;0 + PjPye + p 0 



( 15 ) 



Equations (11) and (14) exhibit the functional relationship between trends in grade 
equivalent and theta variance when the rate of increase in mean 0 is a negatively 
accelerated function of grade. First, Equation (11) shows that the regression of grade 
equivalents (G) on 0 will have a positive P 2 coefficient (.08 in this case). Second, 
Equation (14) shows that if P 2 is positive, grade equivalent variance (a. G ) is bound to 
increase more than theta variance (oy 0 2 ) (assuming there is no decrease in the mean of 
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0 (Hjo))- It is conceivable that Oy G 2 could decrease if a^ 2 were to decrease enough to 
offset the magnitude of (3 2 and the magnitude of change in Pj Q . On the other hand, a- G 2 
could increase even if there were a decrease in Oy 0 2 . 

The within-grade mean, median, and standard deviation of grade equivalents 
were computed using the results of Equations (11) to (15) and the a priori values of 
(Equation (2)) and Oy 0 2 =l for all j. 

Mapping through Integer Assignment. In integer assignment mapping, the equal- 
density points on the 0 scale were boundaries for open intervals within which all thetas 
were mapped to the most probable grade. For example, thetas between 0 3+ 5 and 0 4+ 5 
were mapped to a grade (grade equivalent value) of 4. Thetas below 0 1+ 5 were mapped 
to a 1 (i.e., first grade). Thetas above 0 11+ 5 were mapped to 12. These limits (1 and 12) 
on grade equivalent values were considered too restrictive for grades below 3 or above 
8. Grade equivalent distributions were therefore estimated only for grades 3 to 8. 

For computing the within-grade mean and variance of grade equivalents, the 
weight assigned to a given integer (grade equivalent) was based on the area of the 
within-grade theta distribution over the interval mapped to the given integer. The 
median grade equivalent for grade j was computed as a continuous value using standard 
methods of interpolation. 
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Results 

Before describing the grade equivalent trends, it is important to evaluate and 
compare the performance of the mapping methods. Values for grade equivalent means, 
medians, and standard deviations by grade and mapping method are shown in Table 1. 
Both methods produced medians that were close to the value they would have had if 
the true score procedure of obtaining grade equivalents had been used. 



TABLE 1 

Grade equivalent score distributions by grade and method 





Method of mapping thetas to grade equivalents 






Integer- Assignment 


Quadratic Regression 


Grade 


Std. 

Median Mean Dev. 


Median 


Mean 


Std. 

Dev. 


1 


Not applicable 


1.2 


1.2 


.68 


2 


2.0 


2.0 


.85 


3 


3.0 3.0 1.1 


2.9 


3.0 


1.0 


4 


4.0 4.1 1.2 


3.9 


4.0 


1.2 


5 


5.0 5.1 1.3 


4.9 


5.0 


1.3 


6 


6.0 6.1 1.4 


6.0 


6.1 


1.4 


7 


7.0 7.1 1.5 


7.1 


7.2 


1.5 


8 


8.0 8.1 1.6 


8.2 


8.2 


1.7 


9 




9.2 


9.2 


1.7 


10 


Not applicable 


10.1 


10.2 


1.8 


11 


11.0 


11.1 


1.9 


12 




11.8 


11.8 


2.0 
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With integer assignment, the median for a given grade, j, was within .03 of the intended 
value, j; with quadratic regression, the difference was no larger than .2. The median 
trend plotted in Figure 4 is the intended, linear trend. 

The mapping methods also agreed closely with each other. As shown in Table 1, 
the standard deviations obtained by the method of integer-assignment were within .1 of 
those obtained by quadratic regression~the average absolute difference was only .04. 
Mean values were within .1 of each other, and median values within .2. 



Grade 

equivalent 

score 




Standard 

Deviation 



Grade 



Figure 4. Trends in the median and standard deviation of grade equivalent measures. 



According to both methods of mapping, the within-grade variability of grade 
equivalent scores increased approximately 1.6- fold from grade 3 to 8. Over grades 1 to 
12, to which only the quadratic regression method of mapping was applied, the within- 




19 



15 



grade standard deviation of grade equivalents increased approximately 3-fold, from .68 
to 2.0. This trend is illustrated in Figure 4.^ 

Height Analogy 

Trends in Centimeter Measures 

Height data were obtained from the Fels Longitudinal Growth Study (Wright 
State University School of Medicine, Division of Human Biology). The data consisted 
of the height in centimeters of 212 boys on whom a total of 6,605 measures of height 
were made between the ages of 2.75 and 18.7 years. After editing, the data included the 
height of 160 boys measured within .1 year of their 3rd through 18th birthdays. 
Longitudinal trends in the mean and standard deviation are plotted in Figure 5, and 
corresponding values are given in Table 2. 




Figure 5. Trends in the mean and variance of the height in centimeters. Solid lines are based on Fels data. 
Dashed lines are extrapolations for purpose of constructing age equivalent measures. 
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Certain phenomena that may be considered applicable to cognitive growth, seem 
to account for complex trends in both the mean and variability of height. A collective 
growth spurt appears to start at age 10 and last until about age 14. This accounts for 
positive acceleration in mean height from ages 10 to 14. It seems likely that individual 
differences in the onset of this growth spurt contribute to the large increase in variance 
over this same period. Variance in height is maximum at age 14 because some boys 
have not yet begun their growth spurt, while other boys have reached full adult height. 
After age 14, the rate of growth in the mean is negatively accelerated with age and 
variance decreases as late-starters catch up with those who have reached their maximum 
height. 

Similarly, certain cognitive skills, such as reading, could exhibit a peak in within- 
grade variance during early grades due to large individual differences in the onset of 
development. Many parents teach their children to read two or three years earlier than 
they would otherwise learn in school. This early advantage, however, does not 
necessarily persist into later primary grades, and thus, formal schooling could cause a 
decrease in variance of reading achievement, as measured by multiple choice test 
questions, over time. Other cognitive traits, such as mathematics skill, might not exhibit 
the same trends either because fewer parents teach their children mathematics or because 
the age at which achievement, as measured by multiple choice items, begins to level off 
might be much later for a skill like mathematics than for reading. 

Trends in Age Equivalent Measures 

Centimeter height was mapped to age equivalent height using the method of 
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integer-assignment. This method required extrapolation of centimeter height data for 
ages 1, 2, and 19 to 25. In order to estimate points of equal density between adjacent 
ages, and to assign age-specific weights to age equivalent values (for computing age 
equivalent means, variances, and medians by age), centimeter height was assumed to be 
normally distributed at each age, with means and standard deviations shown in Table 2 
(extrapolated data is not shown). Heights below the equal density point for ages 1 and 
2 were mapped to 1. Heights above the equal density point for ages 24 and 25 were 
mapped to 25. 

TABLE 2 

Distribution of height by age and metric 



Age 






Metric 






Centimeters 




Age equivalents 


Mean 


Std. Dev. 




Median 


Std. Dev. 


3 


95.4 


3.4 




3.0 


.57 


4 


102.7 


3.8 




4.0 


.61 


5 


109.7 


4.3 




5.0 


.69 


6 


116.4 


4.6 




6.0 


.76 


7 


122.7 


5.0 




7.0 


.85 


8 


128.8 


5.3 




8.0 


.95 


9 


134.5 


5.5 




9.0 


1.0 


10 


139.8 


5.8 




10.0 


1.1 


11 


145.0 


6.0 




11.0 


1.1 


12 


150.5 


6.5 




12.0 


1.1 


13 


157.3 


7.5 




13.0 


1.2 


14 


164.7 


7.9 




14.0 


1.8 


15 


171.2 


7.4 




15.1 


3.0 


16 


175.1 


6.7 




16.1 


3.6 


17 


177.1 


6.5 




17.1 


3.8 


18 


178.1 


6.5 




18.0 


3.9 
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Median age equivalent height was within .1 of the corresponding age, as shown 
in Table 2. As shown in Figure 6, growth in the median is practically linear with age, 
as expected. The standard deviation of age equivalent height increases slowly to age 13, 
then increases dramatically. 




5.00 



4.00 



3.00 

Standard 

Deviation 

2.00 



1.00 



0.00 



Figure 6. Trends in the median and standard deviation of age equivalent measures of height. 



Discrepancies between age equivalent and centimeter trends in variability are 
related to nonlinear growth in the mean (in centimeters). When growth in the 
centimeter mean is linear (ages 3 to 10), both metrics show about the same increase in 
standard deviation: the age equivalent standard deviation increases nearly two fold 
(5.8/3.4) as does the standard deviation in centimeters (1.1/.57). When growth in the 
centimeter mean is positively accelerated (ages 10 to 13), the standard deviation in age 
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equivalents increases less than the standard deviation in centimeters (1.2/ 1.1 versus 
7.5/5. 8). When growth in the centimeter mean is negatively accelerated (ages 14 to 18), 
the standard deviation in age equivalents increases (1.8 to 3.9), even though the standard 
deviation in centimeters decreases (7.9 to 6.5). 

Discussion 

This paper provides a compelling demonstration of the arbitrary nature of growth 
trends in cognitive variables. Two metrics, both of which preserve the order of 
performance levels in test data, produced different pictures of cognitive growth. The 
differences were seen to arise strictly from differences in the scaling models. Time- 
indexed measures (by grade or age) will show an inflated rate of increase in variance 
over time relative to an alternative, order-preserving metric that shows negative 
acceleration in the conditional mean over time. From this demonstration, one should not 
expect growth trends in different metrics to look the same. Growth trends in different 
metrics mean different things. It falls to the investigator to carefully consider the 
meaning of scale units and to select the scale that gives growth trends the most useful 
meaning for the problem at hand. 

The unit on the grade equivalent scale is defined by the indexing of performance 
levels on the test to grade levels. Performance levels are indexed to the grade and 
month of the school year at which the level of performance (on the test taken by the 
student) is typical. This indexing system conveys more meaning more clearly to parents 
and teachers of students, particularly at the elementary grades, than probably any other 
type of scale (Hoover, 1984a). It seems reasonable to suppose that trends in growth and 
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variability on the scale may also have practical use in some contexts. 

On other grounds, researchers may take exception to the fact that the grade 
equivalent method of indexing forces the median rate of growth in the norm group to 
be linear. Schulz (1990) argues that a scale is not suitable for studying growth if it 
involves making a priori assumptions about the shape of growth. A scale must be free 
to detect variation in the onset, duration, and intensity of critical periods of growth, and 
the attainment of an asymptote, as were seen in this study with height. These 
phenomena, like the notion of increasing variance with age, are plausible and intuitively 
compelling. This is not to say that a scale should be preferred because it exhibits such 
features. Only that a scale must be free to exhibit such features. 

Another basis on which researchers may find a problem with time-indexed scales 
is shown again by the analogy to height. There were two ways that the variance of age 
equivalent height increased: 1) when age-conditional means of centimeter height became 
more alike (after age 14), and 2) when the variance of centimeter height increased (from 
ages 3 to 10). Likewise, the variance of grade equivalent measures of educational 
achievement can increase by, 1) grades becoming more alike in the behaviors represented 
by the test, and 2) students within grades becoming more different in the behaviors 
represented by the test. In other words, between-group differences are confounded with 
within-group differences. Educational researchers interested in growth will want to be 
aware that time-indexed measures have this potential for confounding. 

The key arbitrary scaling convention in IRT is that the correct response to any test 
item is a function of achievement (0). This assumption specifies a two-way 
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correspondence between numerical scale values (0) and the empirical observations (item 
responses) of the property being measured. This correspondence is one of the criteria 
for representational measurement. Yen (1986) discusses some distinctions between 
representational and index measures (grade equivalents being an index measure) that 
might be of interest to researchers choosing a scale for assessing cognitive growth. An 
illustration of the distinction between these kinds of measurement in this study, for 
example, is that test data and trends in grade-equivalent scores could be generated from 
assumptions about 0; neither test data nor trends in the 0 metric can be derived from 
assumptions about grade equivalent scores. 

The particular form of the item response function is also arbitrary. Lord (1980, 
p84) argues that the logistic function does not necessarily make the 0 metric more 
desirable than other functions. He gives an example of a monotonic transformation of 

if 

0 to 0 . The item response function on 0 is not logistic, but is simple and interpretable. 
The transformation would have no effect on data-model fit or on the representational 
potential of the IRT model, but growth trends in the 0* metric would look quite different 
from those in the 0 metric. Thus, the logistic function is a key ingredient for the shape 
of growth trends when the model is applied to real data. It does not, however, 
determine the shape of growth trends independently of the data. This is an important 
distinction from the a priori linear growth rate of time-indexed scales. 

It seems reasonable to suppose that trends in the log-odds of success, like grade 
equivalent trends, could be useful and practical for some purposes. A given amount of 
change on the 0 scale means there is a corresponding change in the log-odds of success 
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on any given item calibrated to the scale. This study showed that there is a conditional 
relationship between trends in the within-grade variance of number correct scores (for 
on-level tests) and trends in the within-grade distribution of 0. Increasing variability 
on a 0 scale means that differences are increasing among students in terms of their log- 
odds of success on items calibrated to the scale. This correspondence between the scale 
and test data could be an appropriate basis for conclusions about educational programs 
and achievement, particularly when test items sample a criterion domain of 
educationally or socially significant behaviors. 

Based on the demonstration provided by this study, we recommend that when 
discrepancies between growth trends emerge with real data, investigators consider 
whether differences between models, as opposed to estimation problems and technical 
faults could account for the discrepancies. The discrepancies noted between variance 
trends in grade equivalent and IRT metrics (Schulz, et al., 1990; Lee and Wright, 1992) 
are exactly what one would expect if the 0 growth rate of the norm group for the tests 
used in these studies were negatively accelerated. [Growth in the norm group, but not 
necessarily the study group, would have to be negatively accelerated because the grade 
equivalent scores for the study group were norm-referenced.] This seems likely since 
Thurstonian and IRT growth rates for other standardized test batteries are negatively 
accelerated (Yen, 1986), and the mean growth rates for the study groups themselves were 
also slightly negatively accelerated (Schulz, et al., 1990; Lee and Wright, 1992). 

Continued research and refinements of IRT methods and theory are needed to 
tease out how 0 growth trends depend on stage of development, skill, item bank, type 
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of IRT model, and estimation procedure. Trends of decreasing variability in 0 may be 
partially a property of estimation methods (Williams, et al., 1995; Omar, 1996), as 
opposed to a property of the IRT model used. When the within-grade population 
variance of 0 on the NAEP mathematics subtest was estimated directly rather than 
relying on estimated thetas (Camilli, Yamamoto, and Wang, 1993) it increased from 
grade 4 to 8, but decreased from grade 8 to 12. Differences between IRT models may 
also be a factor. The within-grade variability of one-parameter IRT measures of reading 
decreased only slightly across primary grades (Schulz, et al., 1990; Lee and Wright, 1992), 
and the variability of similar measures of mathematics achievement remained constant 
(Lee and Wright, 1992). Becker and Forsyth (1992) found that the within-grade 
variability of one-parameter IRT, three-parameter IRT, and Thurstonian measures of 
performance on an ITED vocabulary test all increased across grades 9 to 12. 

In summary, growth trends based on cognitive test scores are fundamentally 
arbitrary because these scores are ordinal. Since ordinal measurement scales allow such 
a large variety of transformations, compared to metric measurements, one should expect 
to find different shapes of growth functions across time, depending on the scale used— 
and, as we have tried to demonstrate in this inquiry, one should also expect different 
patterns of variances across time depending on the measurement scale. Some cognitive 
scales will show the increasing-variance-with-age trend; other measurement scales for 
the same trait (or even the same test) can be expected to indicate a decreasing-variance- 
with-age trend— or even a constant-variance-with-age pattern. As long as mental traits 
are measured with scores that only rank-order persons, it may well be impossible to 
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determine the 'true' relationship between the age of children and the amount of 
variability in their cognitive performance. 
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Footnotes 

1) If the within-grade distribution of 0 were not normal, the formulas for the median 
(Equation (10)) and equal-density (see text) anchor points in Figure 3 might not be 
precise. (Equation (10) would still be valid for symmetric distributions, where the 
median equals the mean.) However, it seems likely that the true median and equal 
density anchor points would still show substantial positive acceleration, like the anchor 
points in Figure 3, given the negatively accelerated trend in mean 0. The positive 
acceleration is quantified by the p 2 coefficient in Equation (11). The p 2 coefficient is used 
in Equations (13) and (14) to approximate the impact of interpolation on conditional 
grade equivalent distributions, but these equations do not require the corresponding 
conditional 0 distributions to be normal. In mapping through integer assignment, 0 is 
assumed to be normally distributed within grades in order to compute weights for 
integer grade-equivalent values. But in this respect also, it seems unlikely that true 
weights corresponding to reasonable departures from normality would substantially alter 
the results of this study, given the degree of negative acceleration in the equal density 
anchor points of Figure 3. 
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